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Abstract 

This paper considers a class of generalized convex games where each player is associated with a convex objective function, a 
convex inequality constraint and a convex constraint set. The players aim to compute a Nash equilibrium through communi¬ 
cating with neighboring players. The particular challenge we consider is that the component functions are unknown a priori 
to associated players. We study two distributed computation algorithms and analyze their convergence properties in the pres¬ 
ence of data transmission delays and dynamic changes of network topologies. The algorithm performance is verified through 
demand response on the IEEE 30-bus Test System. Our technical tools integrate convex analysis, variational inequalities and 
simultaneous perturbation stochastic approximation. 


1 Introduction 

Recent advances on information technologies facili¬ 
tate real-time message exchanges and decision-making 
among geographically dispersed strategic entities. This 
has boosted the emergence of new generation of net¬ 
worked systems; e.g., the smart grid and intelligent 
transportation systems. These networked systems share 
some common features: on one hand, the entities do not 
belong to a single authority and may pursue different 
or even competitive interests; on the other hand, each 
entity keeps private information which is unaccessi- 
ble to others. It is of great interest to design practical 
mechanisms which allow for efficient coordination of 
self-interested entities and ensure network-wide perfor¬ 
mance. Game theory along with its distributed compu¬ 
tation algorithms represents a promising tool to achieve 
the goal. 

In many applications, distributed computation is exe¬ 
cuted in uncertain environments. For example, mobile 
robots are deployed in an operating environment where 
environmental distribution functions are unknown to 
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robots in advance; e.g., [33,39]. In traffic pricing, pric¬ 
ing policies of system operators may not be available 
to drivers. In optimal power flow control, the structural 
parameters of power systems are of national security in¬ 
terest and kept confidential from the public. The ab¬ 
sence of such information makes game components; e.g., 
objective and constraint functions, inaccessible to play¬ 
ers. Very recently, the informational constraint has been 
stimulating the investigation of adaptive algorithms, in¬ 
cluding [16,21,33] for continuous games and [23,39] for 
discrete games. 

Literature review. Non-cooperative game theory has 
been widely used as a mathematical framework to 
reason about multiple selfish decision makers; see for 
instance [8]. These games have found a variety of ap¬ 
plications in economics, communication and robotics; 
see [3,12,13,16,24]. In non-cooperative games, decision 
making of individuals is inherently distributed. Very 
recently, this attractive feature has been utilized to 
synthesize cooperative control schemes, and a partial 
reference list for this regard includes [5,6,19,33,39]. 

The set of papers more relevant to our work is concerned 
with generalized Nash games where strategy spaces 
are continuous and the actions of players are coupled 
through utility and constraint functions. Generalized 
Nash games are first formulated in [4]. Since then, a great 
effort has been dedicated to studying the existence and 
structural properties of generalized Nash equilibria in; 
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e.g., [31] and the recent survey paper [14]. A number of 
algorithms have been proposed to compute generalized 
Nash equilibria, including ODE-based methods [20,31], 
nonlinear Gauss-Seidel-type approaches [29], iterative 
primal-dual Tikhonov schemes [35] and best-response 
dynamics [28]. 

As mentioned, the set of papers [16,21,23,33,39] inves¬ 
tigates the adaptiveness of game theoretic learning al¬ 
gorithms. However, none of the papers mentioned in 
the last two paragraphs studies the robustness of the 
algorithms with respect to network unreliability; e.g., 
data transmission delays, quantization and dynamically 
changing topologies. In contrast, the robustness has been 
extensively studied for consensus and distributed op¬ 
timization, including, to name a few, [18,27] for time- 
varying topologies, [30] for quantization and [25] for 
time delays. Yet the adaptiveness issue has not been ad¬ 
dressed in this group of papers. 

Contributions. In this paper, we aim to solve a class 
of generalized convex games over unreliable networks 
where the structures of component functions are un¬ 
known to the associated players. That is, we aim to si¬ 
multaneously address the issues of adaptiveness and ro¬ 
bustness for generalized convex games. 

In the games, each player is associated with a convex 
objective function and subject to a private convex in¬ 
equality constraint and a private convex constraint set. 
The component functions are assumed to be smooth and 
are unknown to the associated players. We investigate 
distributed first-order gradient-based computation algo¬ 
rithms for the following two scenarios: 

[Scenario One] The game map is pseudo-monotone and 
the maximum delay (equivalently, the maximum num¬ 
ber of packet dropouts or link breaks) is bounded but 
unknown; 

[Scenario Two] The inequality constraints are absent, 
the (reduced) game map is strongly monotone and the 
maximum delay is known. 

Inspired by simultaneous perturbation stochastic ap¬ 
proximation for optimization in [32], we utilize finite dif¬ 
ferences with diminishing approximation errors to esti¬ 
mate first-order gradients. We propose two distributed 
algorithms for the two scenarios and formally prove their 
asymptotic convergence. The comparison of the two pro¬ 
posed algorithms is given in Section 5.1. The analysis 
integrates the tools from convex analysis, variational 
inequalities and simultaneous perturbation stochastic 
approximation. The algorithm performance is verified 
through demand response on the IEEE 30-bus Test Sys¬ 
tem. A preliminary version of the current paper was 
published in [36] where the adaptiveness issue was not 
investigated. 


2 Problem formulation 

In this section, we present the generalized convex game 
considered in the paper. It is followed by the notions and 
notations used throughout the paper. 

2.1 Generalized convex game 

Consider the set of players V = {1, • • • ,N} where the 
state of player i is denoted as € W C IS,"*. The play¬ 
ers are selfish and pursue different interests. In particu¬ 
lar, given the joint state Xj of its ri- 

valj^ each player i aims to solve the following program 
parameterized by € X-f. 

min s.t. (x^*],x^”*!) < 0, (1) 

xMeXi 

where /, : R” ^ R and : R" ^ R"*’ with n = 
remainder of the paper, we assume that 
the following properties about problem (1) hold: 

Assumption 2.1 The maps fi and G^ are smooth, and 
the maps /i(-,x[“*i) and GW(-,x[“®1) are convex in xW. 
The set Xi is convex and compact, and X f] Y ^ ttl 
where X = Iliey Xi and Y = Higy Yi = {x G 

X 1 GW(x) < 0}. 

We now proceed to provide an equivalent form of prob¬ 
lem (1). To achieve this, we define the set-valued map 
x{ : X-i —>■ 2^' as follows: 

x{ (x[-*i) = {xW e W I gW(xW,x1-*1) < 0 }. 

The set X{ (x^”*!) represents the collection of feasible 
actions for player i when its opponents choose the joint 
state of x^”*] S X-i. With the map a/, problem (I) of 
player i is equivalent to the following one: 

min /i(xl*',x[“‘l). (2) 


Given x^”®! G A_i, each player i aims to solve prob¬ 
lem (2). The collection of such coupled optimization 
problems consists of the generalized convex game (for 
short, CVX). For the CVX game, we adopt the general¬ 
ized Nash equilibrium (for short, GNE) as the solution 
notion which none of the players is willing to unilaterally 
deviate from: 


^ We use the shorthand —i = V\ {i} throughout the paper. 
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Definition 2.1 The joint state x & X C]Y is a general¬ 
ized Nash equilibrium of the CVX game if the following 
holds: 

Mi) < V:eW e X {Vi € K 


Denote by Xcvx the set of GNEs of the CVX game. The 
following lemma verifies the non-emptiness of X^vx- 

Lemma 2.1 The set of generalized Nash equilibria of 
the CVX game is not empty, i.e., Xcvx 0- 

Proof: Recall that fi is convex and V n V is compact. 
Hence, Xcvx 7 ^ 0 is a direct result of [14,31]. • 

In the CVX game, the players desire to seek a GNE. 
It is noted that fi, and Xi are private informa¬ 
tion of player i and unaccessible to others. In order to 
compute a GNE, it becomes necessary that the players 
are inter-connected and able to communicate with each 
other to exchange their partial estimates of GNEs. The 
interconnection between players will be represented by 
a directed graph Q = {V,£) where £ VV x V \ diag(V) 
is the set of edges. The neighbor relation is determined 
by the dependency of fi and/or G^ on In particu¬ 
lar, {i,j) G f if and only if fi and/or G^*! depend upon 
Denote by — {/ € V | {i,j) € £} the set of 
in-neighbors of player i. 

In this paper, we aim to develop distributed algorithms 
which allow for the computation of GNEs in the presence 
of the following two challenges. 

(1) Data transmissions between the players in V are un¬ 
reliable, and subject to transmission delays, packet 
dropouts and/or link breaks. We let x'^^^k — rf'{k)) 

with (k) > 0 be either (z) the outdated state of 
player j received by player i at time k due to trans¬ 
mission delays; or [ii) the latest state of player j re¬ 
ceived by player i by time k due to packet dropouts 
and/or link breaks. For example, if the link from 
player j to player i is broken at time k, then player i 
cannot receive the message xl'll(fc) from player j. 
For this case, player i uses the more recent received 
message sent from player j which is x^^^ {k — (fc)). 
Denote by Mik) = {x^^\k — Tj^\k))}j^j,^m the set 
of latest states of A/™ received by player i at time k. 
Let Tinax — sup;.>Q max^gy maXjg_,yiN (k) be the 
maximum delay or the maximum number of suc¬ 
cessive packet dropouts. 

(2) Each player i is unaware of the structures of fi and 
G1®1 but can observe their realizations. That is, if the 
players input the joint state x into fi, then player i 
can observe the realized value of fi{x). 


The disclosure of the value fi{x) is case dependent. 
In [33,39], mobile sensors are unaware of environmen¬ 
tal distribution functions but they can observe induced 
utilities via on-site measurements. This is an example of 
engineering systems. Section 6 will provide a concrete 
example of demand response of power networks where 
the system operator discloses realized values via com¬ 
munication given the decisions of end-users. This is an 
example of social systems. 

3 Preliminaries 

In the CVX game, each player is subject to an inequal¬ 
ity constraint. In optimization literature, Lagrangian re¬ 
laxation is a widely used approach to handle inequal¬ 
ity constraints. Following this vein, we will perform La¬ 
grangian relaxation on the CVX game and obtain the 
unconstrained convex (for short, UC) game. 

In the UC game, there are two sets of players: the set 
of primal players V and the set of dual players Vm = 
{I,-- - ,N}. Define the following private Lagrangian 
: R" X ^ R: = Mx) + (x)), 

for primal player i G V and dual player i G Vm- 
Given any G X-i and G R>q, each primal 

player i G V aims to minimize TLi over x^*] G Xp, i.e., 
min,j,[i]gj(-.and, instead, the objec¬ 
tive of the dual player i G Vm is to maximize Tdi over 
G Mi C R>/; i.e., min^[i]gM. —'Hi(x,where 
the set Mi C R™/ is convex, non-empty and will be 
introduced in the sequel. We let rj = (x, p.) and the set 
K = X X M with M = Yiiev above game 

among the players in U U is referred to as the UC 
game parameterized by the set M. The set of M will play 
an important role in determining the properties of the 
UC game and we will discuss the choice of M later. The 
solution concept for the UC game parameterized by M 
is the standard notion of Nash equilibrium given below: 

Definition 3.1 The joint state of {x,fi) G X x M is 
a Nash equilibrium of the UC game parameterized by 
M if the following holds for all i G V: ?j.i(x,M^) < 
'Hi(xW, x[“*l,VxW G Xi, and the following holds 
for all i G Vm-' TLiix, < 'Hj(x, G My 

Denote by X\jc{M) the set of NEs of the UC game pa¬ 
rameterized by M. We now proceed to illustrate the re¬ 
lation between the UC game and the CVX game. Before 
doing so, let us state the following boundedness assump¬ 
tion on the dual solutions: 

Assumption 3.1 There is a vector § = (djjjg y G R>o 
such that for any (x,fi) G Xuc(R>o)> ^ for 

iGV. 

Remark 3.1 We would like to make a remark on As¬ 
sumption 3.1. For convex optimization, it is shown in [17] 
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that the Lagrangian multipliers are uniformly bounded 
under the standard Slater’s condition. This boundedness 
property is used in [26] and further in [37,38] to solve 
convex and non-convex programs in centralized and dis¬ 
tributed manners. In this paper, Assumption 3.1, on one 
hand, ensures the existence of GNEs, and on the other 
hand, guarantees the boundedness of estimates and gra¬ 
dients for the case of unknown Tmax- We will discuss the 
verification of Assumption 3.1 in Section 5. • 

The following proposition characterizes the relations be¬ 
tween the UC game and the CVX game. 

Proposition 3.1 The following properties hold: 

(PI) Consider any {x, fi) € Xuc(M). Wehavex € Xcvx 
if the following properties hold: 

• feasibility; i.e., G^’'\x) < 0 for all i G V; 

• slackness complementarity; i.e., (x)) = 0 

for all i GV. 

(P2) Suppose Assumption 3.1 holds. Consider the UC 
game parameterized by M with Mi = {p)A g 
I ^ + ^i} for some ri > 0. Then 

X\jc(M) 0. In addition, for any G 

XuciAI), it holds that x G Xcvx- 

Proof: The proof of (PI) is a slight extension of 
the results in [9] to our game setup. For the sake of 
completeness, we summarize the analysis here. Since 
(/iW,GW(i))=0, we have the following relation: 

fi(x) = 9{i(x,[i^’^) - (3) 

Since (x, fi) G Xuc(-^), then the following relation holds 
for all G Xp. 

'Hi{x, < 0. (4) 

Substitute (3) into (4), and it renders the following for 
any G Xp 

Mi) < /,(xW,x[-*l) + (/fW,GW(xW,x[-*l)), 

and thus fi{x) < fi{x^'‘\x^~’‘^), Vx^*] G x/(xl“*]), 

by noting that (/fW,GW( xW, x[“*l)) < 0 for any 

xW G xl (x[“*l). Recall that GW(x) < 0 for all i G V. 
The above arguments hold for all i G V, and thus it 
establishes that x G Xcvx- 

We now proceed to show (P2). Since > 0, Hi is a 
positive combination of convex functions of x^*!. Thus 
Hi is convex in x^. It is easy to see that —Hi is con¬ 
vex (actually affine) in x^®!. Since X and M are convex 
and compact, it follows that X\jc{M) ^ 0 by the re¬ 
sults on generalized convex games in; e.g., [14,31]. Pick 


any (x,/i) G Xuc(-^) and then ||/il®]|| < di by Assump¬ 
tion 3.1. We then have the following relation: 

H^{i,[lM - Hi{x,pLM > 0, G Mi, 

and thus, 

G[®1(x)) < 0, V/x[®l G M,. (5) 


We now proceed to verify by contradiction the feasi¬ 
bility of GW( x) < 0 and the complementary slackness 
of (/il®l,G[®1 (x)) = 0. Assume that GW(x)f > 0. We 

choose such that -\- nn and = 0 

for i' i where tt > 0 is sufficiently small such that 
fjjA g Mi. Then it follows from (5) that riGi^{x)i < 0 
which is a contradiction. Hence, we have the feasibility 
of G[®1 (x) < 0. Combine it with fM > 0, and it ren¬ 
ders that (/iW), G[®1 (x) < 0. On the other hand, we let 
pfA = 0 in the relation (5) and have (/iW,GW(x)) > 0. 
Hence, it renders that (/iW,GW(x)) = 0. We reach the 
desired result by using the property (PI). • 

3.1 Notations and notions 

The vector 1„ represents the column vector with n ones. 

m 

The vector ek is the one in R" whose Gth coordinate 
is one and whose other coordinates are all zero. For 
any pair of vectors a,b G R^, the relation a < b means 
Of, < bi for a\\ 1 < (. < p. Since the function fi is con¬ 
tinuous and Xi is compact, then the following quanti¬ 
ties are well-defined: cr^min = inf^ex, fi{x), cr^max = 

/*(a;), CTmin = ini,^cxY.iCV fM), O-max = 

fi(^)- Given a non-negative scalar se¬ 
quence {Q;(fc)}fc>o, it is summable if ^ 

and square summable if YfkLo oA.k)'^ < -foo. 

In the remainder of the paper, we will use some notions 
about monotonicity, and the readers are referred to [7,15] 
for detailed discussion. The mapping F : Z ^ Z' is 
strongly monotone with constant p > 0 over Z if for each 
pair of rj, r]' G Z, the following holds: 

(F(?7) - F{r]'), ri-r]')> p\\ri -p'\\^. 

The mapping F : Z ^ Z' is monotone over Z if for each 
pair of rj, p' G Z, the following holds: 

{F{r]) - F{ri'),r]-r]') > 0. 

The mapping F -. Z ^ Z' is pseudo-monotone over Z if 
for each pair of rj, rj' G Z, it holds that {F{rj'),rj — rj') > 0 
implies {F(rj), rj — rj') > 0. It is known that strong mono¬ 
tonicity implies monotonicity, and monotonicity implies 
pseudo-monotonicity [15]. 
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Given the non-empty, convex, and closed set Z S R", 
the projection operator onto Z, Pz : R" —t Z, is defined 
as Pz[z\ = argmin,j,g 2 l|a: — z\\. The following is the non- 
expansiveness of the projection operator; e.g., [9]. 

Lemma 3.1 Let Z be a non-empty, closed and convex 
set in R". For any z € R", the following holds for any 
y^Z:\\Px[z]-yr<\\z-yr-\\Px[z]-zr. 

We define V as the gradient of the convex 
function at and V^[i]'Hi(x,= 

GW(x) as the gradient of the concave (actually affine) 
function 'Hi(x, •) at Define V0 as the map of partial 
gradients of the players’ objective functions: 

V0(?7) = [V,[iiHi(x, /rW)^ • • • 

The map V0 is referred to as the game map. It is well- 
known that V0 is monotone (resp. strongly monotone) 
over K if and only if its Jacobian V^0(?7) is positive 
semi-definite (resp. positive definite) for all p G K. 

For G[*l (x) < 0 being absent, we define V,j,[i]/i(x) as 
the gradient of the convex function /i(',x[“*l) at xl*l. 
Define V0’’ as the map of partial gradients of the players’ 
objective functions: 

V0’'(x) ^ [V,m/i(x)^ • 

The map V0’' is referred to as the (reduced) game map 
for GW(x) < 0 being absent. 

Given any convex and closed set Z, let be the pro¬ 
jection operator onto the set Z. 

The following lemma is a direct result of the smoothness 
of the component functions and the compactness of the 
constraint sets. 

Lemma 3.2 The image set of the game map V0 zs uni¬ 
formly bounded over K. In addition, the game map V0 
(resp. VG*" j is Lipschitz continuous overK with constant 
Lq (resp. L^r-). 

4 Distributed computation algorithms 

In this section, we will study the scenarios mentioned in 
the introduction. Proposition 3.1 reveals that it suffices 
to compute a Nash equilibrium in Xyc if Assumption 3.1 
holds. In the remainder of this section, we will suppose 
Assumption 3.1 holds, and then each dual player i can 
define the set of Mi A {^W g R™* | ||^W|| < i?i-|-ri} with 
Ti > 0. Assumption 3.1 will be justified in Section 5.2. 


4-1 Scenario one: pseudo-monotone game map and un¬ 
known Tniax 

In this section, we synthesize a distributed first-order al¬ 
gorithm for the case where the quantity Xmax is unknown 
and the game map V 0 is merely pseudo-monotone. 

Algorithm 1 is based on projected primal-dual gradient 
methods where each primal or dual player updates 
its own estimate by moving along its partial gradient 
with a certain step-size and projecting the estimate onto 
the local constraint set. Recall that fi and G^*! are un¬ 
known to player i. Then player i cannot compute partial 
gradient Hi. Inspired by simultaneous perturbation 
stochastic approximation in; e.g., [32], we use finite dif¬ 
ferences to approximate the partial gradient i.e., 

foiM = 1 , • • • ,ni, 

« ^^^('Hz(xW(A:) -kCi(fc)e|f|,x["*](fc),/rW(A:)) 

- 'H,(xW(fc) - Ci(fc)e[f],x["*l(fc),/rW(fc))), (6) 

where—Ci(fc)en] andci(fc)e|f] are two-way perturbations. 
In addition, V^[i]Hi(x(A;),^W(fc)) = GW(x(fc)). In Al¬ 
gorithm 1, Vl^ik) (resp. 'Df}{k)) is the right-hand side 
of ( 6 ) (resp. V^[i]'Hi(x,^W)) with delayed estimates. 
Here we assume that player i can observe the values as¬ 
sociated with fi and G^ and thus Vx^k) and I?|)^(fc). 

The scalar Ci{k) > 0 is the perturbation magnitude. 
When it is small, the finite-difference approximation is 
close to the partial gradient. In order to asymptotically 
eliminate the error induced by the finite-difference ap¬ 
proximation, Ci{k) needs to be diminishing. The decreas¬ 
ing rate of Ci(k) should match that of computation step- 
sizes a{k). Otherwise, the convergence to Nash equilib¬ 
rium may be prevented. This is captured by (A4) of The¬ 
orem 4.1. 

The following theorem demonstrates that Algorithm 1 
is able to achieve a GNE from any initial state in K. 

Theorem 4.1 Suppose the following hold: 

(Al) the quantity T^ax is finite; 

(A2) Assumptions 2.1 and 3.1 hold; 

(AS) the sequence of{cx{k)} is positive, not summable but 
square summable; 

(A4) the sequence of {a{k) maxCi{k)} is summable; 

iav 

(A5) the game map V0 is pseudo-monotone over K. 


^ In practical implementation, the update rules of primal 
player i and dual player i are both executed by real player i. 
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Algorithm 1 Distributed gradient-based algorithm for 

Scenario one _ 

Require: Each primal player i G V chooses the initial 
state a:[*l(0) € Xi. And each dual player i G Vm defines 
the set Mi and chooses the initial state /rl*l(0) G Mi. 
Ensure: At each k > 0, each player in E U Vm executes 
the following steps: 

1. Each primal player i G V updates its state according 
to the following rule: 

XW (fc -f 1) = Px. {k) - a{k)vl^^ (fc)], 

where the approximate gradient 'Dx\k) is given by 

+c.(fc)eM,A,(fc)) 

+ (/X W (k) , G W (x W (fc) + c, (fc)e[fl, A, (fc))) 

- - Ci(fc)e^], Aj(A:)) 

- (/xW(^),GW(xW(fc) - c,(fc)eM, A,(fc)))}, (7) 

for t' = 1, • • • , rii. 

2. Each dual player i G Vm updates its state according 
to the following rule: 

^^^{k + 1) = PM.[A^'*’(fc) + oc{k)Vf{k)], 

where the gradient is given by = 

A^(A:),AxW(fc)) = A^(A:)). 

3. Repeat for fc = A: -I- 1. 


For any initial state 77 ( 0 ) € K, the sequence of {ri{k)} 
generated by Algorithm 1 converges to some {x, fi) G Xyc 
where x G Xcvx- 


Proof: Eirst of all, it is noted that K is compact since 
each Xi and Xi is compact in Assumption 3.1 and (P2) 
in Proposition 3.1 as well as V0 is uniformly bounded 
over K in Lemma 3.2. 


Then we write Algorithm 1 in the following compact 
form: 


'q{k + l) =fK[r]{k) - a{k)V{k)\, (8) 


where T>(fc) = {{T)f\kY)J^y, {k )'^is sub¬ 

ject to time delays and perturbations. Then pick any 
7] G K. It follows from the non-expansiveness of the pro¬ 
jection operator Px, Lemma 3.1, that for any rj G K, 


the following relation holds: 

||? 7 (fc-kl)- 77 f 

<Mk)-c^ikMk)-rjr-\\V{k)r 

< \\r]{k) - a{k)V{k) - r]\\^ 

= Mk) - bf - 2a{k){V{k),r]{k) - rj) 

+ a{knVik)r. (9) 

It follows from (9) that 

2a{k){V{k),r]{k) - ij) < Wvik) - vf 

-h(fc + l)-r?f+ a(A:f||P(fc)f. (10) 

Eor any fc > 0 and i, we define the following: 

pW(fc) = 

V{k) 4 iiV^Hkf)ly,{-Vlii{kf%yJ^, (11) 

which are the gradients evaluated at the delay-free 

states. So, the quantity 'D{k) is free of time delays and 
perturbations. 

Similarly, for any k > 0 and i, we define the following: 

pW(A:) = XxMx^^{k),Uk),ii^^{k)), 
i’li'ik) = V^,iH.(a:W(fc), A,(fc),MW(fc)), 

V{k) 4 mHkf)ly,i-V]^^ikf)lyJ^, 

which are the gradients evaluated at the delayed states. 
Then the quantity P(A:) is free of perturbations but sub¬ 
ject to time delays. 

With the above notations at hand, the relation (10) im¬ 
plies the following: 

2aik){f>{k),rjik) - 77) < a{knVik)r + Uk) - bf 

- Wvik + l) - vf + 2a{k){T>{k) - V{k),r){k) - 77) 

-I- 2a{k){V{k) — 'D{k),r]{k) — 77) 

<a{krmk)r+uk)-vr 

- Wvik + 1) - bf + 2a{k)\\V{k) - X>(A:)|||| 77 (A:) - 77 II 

+ 2a(fc)|jP(fc)-P(fc)||||r7(fc)-r7||. (12) 

Now let us examine the term with ||T’(A:) — T>{k)\\ on the 
right-hand side of (12). Since x{k) G X and X is convex 
and closed, it then follows from the non-expansiveness 
of the projection operator Px, that 

|| 77 (fc + 1)- r]{k)\\ = ||Px[b(*) - a{k)V{k)] - Px[ 77 (fc)]|| 
<a{k)\\V{k)\\. (13) 
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Consequently, it follows from the Lipschitiz continuity 
of the game map V0 and (12), (13) that 

\\V{k)-V{k)\\ 

< - ^W(fc)ii + \\v\^{k)-f>^;^ik)\\) 

iGV 

k-l 

<2NLb \\v{t + 1) - vir)\\ 

T — k Tin ax 

k-l 

< 2 NLe T. »MIPMI|. ( 14 ) 


Since K is compact and the image of V0 is uniformly 
compact, (14) implies that there is T > 0 such that 

2aik)\\V{k)-V{k)\\Uk)-v\\ 

k-l 

<2NLe Y 2 Q;(fc)Q;(T)||X>(T)||||? 7 (fc) - 77 II 

T — k Tmax 

k-l 

< 2 NLe Y + 

T — k Tmax 

k-l 

< 2NLQT^^^a{kf + T E “M"- (15) 


Now consider the term with ||I?(fc) — 'D{k)\\ on the right- 
hand side in (12). Recall that K is compact. By the 
Taylor expansion, we reach that 

2?i?'(A)=V,,i/.(xW(fc),A,(A))) 

rrii 

e=i 

(16) 


With the above relation, we have the following for 

2?W(fc) -X>W(fc): 

=0, pt?l(fc)-I?iil(A)) =0(c,(fc))l„.. 

(17) 


following estimate holds: 

T 

2Yot(k){V{k),ri{k) - rj) 
k^Q 

T 

< ||?7(0) - ? 7 f - ||? 7 (T -f 1) - r/f + T' ^ a{kf 

k^O 

T 

+ T" a{k) maxci(fc). (18) 

fc=o 


Since {a(fc)^} and {a{k) maxi^v Ci{k)} are summable, 
then the right-hand side of (18) is finite when we let 
T —>■ -|-(X). We now show the following by contradiction: 

liminf(I?(A:),? 7 (A:) — ry) < 0. (19) 

Ai—>-+oo 

Assume that there are ko > 0 and e > 0 such that 
{V{k),r]{k) — rj) > e for all k > ko. Since a{k) > 0 and 
{a{k)} is not summable, then we have the following: 

+00 

Y(^i.k){V{k),ri{k) - rj) 
k^O 

kp 

> Y<^ik){V{k),r]{k)-r]) 

k^O 

+00 

+ Y <^ik){V{k),ri{k)-rj) 

k—kp-\-l 

kp +00 

>Y<^ik){'D{k),r]{k)-r]) + e Y 

k —0 fc—fco + 1 

> -foo. 

We then reach a contradiction, and thus (19) holds. 
Equivalently, the following relation holds: 

limsup(2?(A:),ry — r]{k)) > 0. 

Ai—>-+oo 

Since V0 is closed, the above relation implies that there 
is a limit point 7 ) G K oi the sequence {?y(A:)} such that 
the following holds: (V0(ry), ?y —?y) > 0, V?y G K. Since 
V0 is pseudo-monotone, then we have the following re¬ 
lation: 

(V0(?y),ry-?y) > 0, Vry G AT. (20) 


Notice that 1127(^)11 is uniformly bounded. Substi¬ 
tute (15) and (17) into (12), sum over [0,r], and it 
renders that there is some T', T" > 0 such that the 
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We now set out to show the following by contradiction: 

(V0(?y),ry-?y) > 0, V?y G (21) 



Assume that there is r) G AT such that the following holds: 

{veiv),f}-fj) <0. (22) 

Now chooser G (0,1), and define r/g = 'q+e{f} — f}). Since 
K is convex, then we have 7]^ G K. The following holds: 

(V0(?7e), (1 - e){ji - ?))) = (V0(?7e), ?7e - ?)) > 0, 

(23) 

where in the inequality we use (20). It follows from (23) 
that the following relation holds for any £ G (0,1): 

(V0(77,),7)-7)) >0, VryGK. (24) 

Since V0 is closed, letting e —)• 1 in (24) gives that: 
(V0(r7), ?7 — ?)) > 0, which contradicts (22). As a result, 
the relation (21) holds. By [28], the relation (21) implies 
that f] G Xuc- We replay r] by 'q{k) in (21), and establish 
the following: 

(V0(7)),r7(A:)-r7)>O. (25) 

Since V0 is pseudo-monotone, it follows from (25) that 
(P(A:),r;(fc)-7)) >0. (26) 


convergence rate. The shortcoming can be partially ad¬ 
dressed by choosing a constant step-size when the in¬ 
equality constraints are absent and (A5) in Theorem 4.1 
is strengthened to the following one: 

Assumption 4.1 The map V0’’ is strongly monotone 
over X with constant p. 

Algorithm 2 The distributed gradient-based algorithm 

for Scenario two _ 

Require: Each player in V chooses the initial state 
xW(0) G W. 

Ensure: At each A: > 0, each player i G V executes the 
following steps: 

1. Each player i G V updates its state according to the 
following rule: 

xW(fc -f 1) = Pjf,[xf*l(fc) - aD^^ik)], 

where the approximate gradient I?[*l(fc) is given by: 

[D^Hk)V = +c.(fc)eM,A,(A:)) 

- - c,(fc)e](], A,(fc))}, 

for A = 1, • • • ,ni. 

2. Repeat for fc = fc -I- 1. 


Replace rj with fj in (10), apply (26), and it renders: 

Uk + 1) - ryf - ||r?(fc) - r?f < a(fc)2||I?(fc)f. (27) 
Sum up (27) over [ 5 , k] and we have 

k-l 

Mk) - fyf < Ms) - vf + ( 28 ) 


Algorithm 2 is proposed to address Scenario two. In par¬ 
ticular, Algorithm 2 is similar to Algorithm 1, but has 
two distinctions. Firstly, Algorithm 2 excludes the in¬ 
equality constraints. Because the game map V0 cannot 
be strongly monotone due to TLi being linear in . Sec¬ 
ondly, thanks to the strong monotonicity of V0, a con¬ 
stant step-size replaces diminishing step-sizes. The fol¬ 
lowing theorem summarizes the convergence properties 
of Algorithm 2. 


We take the limits on k first and then s on both sides 
of (28). Since {a{k)} is square summable and {I?(fc)} is 
uniformly bounded, we have 

limsup ||? 7 (A:) — < liminf ||77(s) — 

k^oo 


It implies that {\\r]{k) — i)]]} converges. Since fj is a, limit 
point of {ri{k)}, {r]{k)} converges to ry G Xuc- Further¬ 
more, we have x G Xcvx by (P2) in Proposition 3.1. • 


Theorem 4.2 Suppose the following holds: 


(Bl) 

(B2) 

(BS) 

(B4) 

(B5) 


the quantity 0 < Tmax < known; 

Assumptions 2.1 and 3.1 hold; 

the step-size a G (0, )f 

The sequence {max^gu Ci(fc)} is summable. 
Assumption 4-1 holds. 


For any initial state a:(0) G X, the sequence of {x{k)} 
generated by Algorithm 2 converges to x G Xcvx- 


4.2 Scenario two: strongly monotone reduced game map 
and known Tmax 

In the last section, the convergence of Algorithm 1 is en¬ 
sured under the mild assumptions: Tmax is unknown and 
the game map V0 is merely pseudo-monotone. This set 
of assumptions requires the usage of diminishing step- 
sizes. Note that diminishing step-sizes may cause a slow 


Proof: First of all, for any k > 0 and i, we de¬ 
fine X>W(A:) = V,j,w/i(a;W(A:),{xbl(fc)}j-g_ViN), which 
is the gradient evaluated at the delay-free states. So, 
the quantity P(fc) is free of time delays and pertur¬ 
bations. Similarly, for any fc > 0 and i, we define 
p[*l(fc) = X^[i] fi{x^^\k), Ai{k)) which is the gradient 
evaluated at the delayed states. Then the quantity P(fc) 
is free of perturbations but subject to time delays. 



We then write Algorithm 2 in the following compact 
form: 

x{k + 1) =f‘x[x{k) — aD{k)]. (29) 

It is noticed that x G Xcvx is a fixed point of the op¬ 
erator Pjf [• — aV6’’(-)] for any a > 0; i.e., it holds that 
X = Px [x — aV 0’’ (i)]. By this, we have the following 
relations: 

lk(fc+ 1) - if 

= ||Pxf (A:) - aD{k)] - Px[i - aVOf i)]f 

< \\{xik) -x)- a{D{k) - V0fi))f 

= ||a:(fc) — if — 2a{x{k) — i, D{k) — V0’'(i)) 

+ a^\\{D{k) - b{k)) + {b{k) - b{k)) + {b{k) - w)f 

— 2a{x{k) — X, D{k) — b{k)) 

— 2a{x{k) — X, b{k) — b{k)) 

< (1 — 2pa -i 4LQ,.a^)||a:(fc) — if 

+ AaW\D{k)-b{k)r + \\b{k)-b{k)r) 

+ 2a\\x{k) - i||(p(fc) - ^(A:)|| + \\b{k) - b{k)\\), 

(30) 

where we use the non-expansiveness property of the pro¬ 
jection operator Px in the first inequality, and the strong 
monotonicity and the Lipschitz continuity of V0’’ in the 
last inequality. For the term of \\b{k) — b{k)\\ in (30), 
one can derive the following relations: 

fc-i 

||L)(A:) - I)(fc)|| < iVLer- ||x(r-il)-a;(T)|| 

T — k 

k — 1 

<NLer- Y (If(T-il)-ill-h||a;(r)-i||) 

T = k 

k -1 

<2NLe. Y (31) 


where we use the relation 2ah < -i 5^. 

To study the convergence of x{k) in (33), we define the 
following notation: 

ipik) = xik) - X, ||x(fc)|| = max ||V'(t)||, 

«-Tmax<T<fc-l 

e(fc) = 4Q;^(maxci(fc))^ -|- 2amaxci(fc)||a;(fc) — i||. 

iGV iGV 

Then (33) is rewritten as follows: 

U{k -i 1)11 < a||V'(fc)|| -i 6||x(A:)|| + e{k). (34) 

where a = (1 — 2pa + 4L0rQ;^ -i 4A^Te''<a'rmax), b = 
Tmax(32iV^T|,,.a^ -I- ANLqtq) and the sequence {e(A:)} 
is diminishing. From the recursion of (34), we can derive 
the following relation for any pair of fc > s > 0: 

k-1 k-1 

||V'(A:)|| < (s)|l + Y + Y ®(^) 

T—S T—S 

k-1 k-1 

< (s)||+& sup ||x(r)||^a'=-^-f ^e(T) 

h ^ 

< (s)||-h - sup ||x(t)||+X1®('^)- (35) 

Recall that {max^gy Ci(fc)} and thus {(max^gy cffc))^} 
are summable. Take the limits on k and s at both sides 
of (35), and it gives the following relation: 

limsup ||i/'(fc)|| < ——limsup ||x(A:)||. (36) 

fc—>-+00 -L Oj ;2_>._|_oo 

On the other hand, one can see that 

limsup ||x(A:)|| = limsup ||■i/'(fc)||. (37) 

fc—>-+oo fc—>-+oo 


Similar to (17), we have 

3^{k) - D^^{k) = 0{c,{k))U^. 


(32) 


Since a G (0, ) > l^en ^ < 1. The com¬ 

bination of (36) and (37) renders that 


Substituting (31) and (32) into (30) yields: 
||x(A:-kl)-if 

< (1 - 2 pa + + 4A^Lerarmax)||a;(fc) - if 

k-1 

+ {32N'^LQ^a^ + 4:NL0Ta) 'Y If ('^) — 

T — k Tin ax 

-I- 4a^(maxci(fc))^ -|- 2amaxci(fc)||x(fc) — i||, (33) 

i^V iGV 


limsup ||V'(fc)|| = 0. (38) 

fc—>-+oo 

Apparently, 

liminf ||i/'(fc)|| > 0. (39) 

fe—>-+oo 

The combination of (38) and (39) establishes the con¬ 
vergence of {x{k)} to X. It completes the proof. • 
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5 Discussions 


5.1 Comparison of two scenarios 

The two proposed algorithms are complementary. Al¬ 
gorithm 1 can address inequality constraints, does not 
need to know Tmax and merely requires the game map to 
be pseudo monotone. It comes with the price of poten¬ 
tially slow convergence due to the utilization of dimin¬ 
ishing step-sizes. In contrast, Algorithm 2 cannot deal 
with inequality constraints, needs to know Tmax and re¬ 
quires the game map to be strongly monotone. It comes 
with the benefit of potentially fast convergence due to 
the utilization of a constant step-size. 

5.2 Discussion on Assumption 3.1 

The proposed algorithms rely upon Assumption 3.1. In 
what follows, we will provide two sufficient conditions 
for Assumption 3.1. Recall that CTi^max, CTmin and 

CTmax are defined in Section 3.1. 

5.2.1 Global Slater vectors 

Assumption 5.1 There exist x G X and ct' > 0 such 
that 11 — GW(x)||oo > o"' for all i gV. 

The vector x that satisfies Assumption 5.1 is referred to 
as the global Slater vector. The existence of the global 
Slater vector ensures the boundedness of Xuc{M) as 
follows. 

Lemma 5.1 If Assumption 5.1 holds and is con¬ 
vex in X, then for any {x,fi) G Xuc(E->o); holds that 

< ^max-g-min j 

Proof: Pick any fj = {x, fi) G Xuc(E->o)- H holds that 
(V0(?7),77-r/) <0. (40) 

Choose rj = (x^, 0^)^ in (40), and we have 

0> (V0’'(i),x-x)-^(/i[*l,GW(x)) 

i&V 

rrii 

iev 1=1 

> (V0’'(x),x-x)-^(/i[^GW(x)), (41) 

iev 


where the last inequality uses /i) > 0 and Taylor theo¬ 
rem and the convexity of g[*^ in x. It follows from (41) 
and Assumption 5.1 that 

-(V0'’(x),X - X) ^ Q-max - O'min 



5.2.2 Private Slater vectors 

In Lemma 5.1, each player has to access the global in¬ 
formation of x, a', (Tniin and (Tmax- In what follows, we 
will derive a sufficient condition which only requires pri¬ 
vate information of each player i to determine an upper 
bound on . 

Assumption 5.2 For each i G V, there exists (Ji > 
0 and Xi G Xi such that for any {x,fi) G Xyc; II ~ 
GW(x[*1,x[“®1)||oo > cTi holds. 

The vector x^ that satisfies Assumption 5.1 is referred 
to as the private Slater vector. One case where Assump¬ 
tion 5.2 holds is that G^*! only depends upon x^ and, for 
eachf € R, there is x[*l € such that || — G[*1 (xW)||oo > 
<Ji holds for some > 0. 

Lemma 5.2 If Assumption 5.2 holds, then for any 
{x,fi) G Xuc(]II>o)> it holds that ||/if*]||oo < 
for i G V. 

Proof: Pick any {x,fi) G Xuc(IR->o)- Hy Assump¬ 
tion 5.2, there is x^*! G Xi such that 

||-Gl*l(xW,x[-*l)|U >a. 

Since (x, fi) G XyC) we then have the following: 

<H.(x[*l,x[-*l,/i''’) 

where in the first inequality we use 0 G Mi and in the 
second inequality we use x^*! G Xi. The above relations 
further render the following: 

ftix) < /i(xW,xl"*') -b (/iW,GW(xW,x("*l)). (42) 

Recall ||—GW(x1*1,x[“*1)||oo > cry Then the relation (42) 
implies the relation of ||/i^*'l|oo < , 

5.3 Future directions 

The current paper imposes several assumptions: (1) the 
compactness of Xi; (2) the smoothness of component 
functions; (3) identical a{k) or a for all the players; (4) 
the convexity of component functions and constraints. 
In particular, the compactness of Xi ensures the uniform 
boundedness of partial gradients. The convexity guar¬ 
antees that local optimum are also globally optimal in 
the sense of Nash equilibrium. It is of interest to relax 
these assumptions. 
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6 Numerical simulations 

In this section, we will provide a set of numerical sim¬ 
ulations to verify the performance of our proposed al¬ 
gorithms. The procedure sample(A) returns a uniform 
sample from the set A. 

6.1 Algorithm 1 


The above description defines a non-cooperative game 
among end-users in Vi as follows: 

min CiRi -t- R ^) Ri^ Ui (^Li Ri^ 

Ri G [0,Z/i] 

S.t. ij^^i (L — i?) < 

HgS-Hi{L-R)-<Q 
- HgS + Hi{L - R) - < 0. (45) 


Consider a power network which can be modeled as an 
interconnected graph Qp = {Vp,£p} where each node 
i G Vp represents a bus and each link in £p represents 
a branch. The buses in Vp are indexed by 1, • • • ,N and 
bus 1 denotes the feeder which has a fixed voltage and 
flexible power injection. A lossless DC model is used to 
characterize the relation between power generations and 
loads at different buses and power flows across various 
branches. Each bus is connected to either a power load 
or supply and each load is associated with an end-user. 
The set of load buses is denoted by Vi C Vp and the set 
of supply buses is denoted by Vs C Vp. 

The maximum available power supply at bus i S Vs is 
denoted by 5^ > 0, and the intended power load at bus 
f S Vi is denoted by Li > 0. If the total power supply ex¬ 
ceeds the total intended power load, then all the intended 
loads can be satisfied. Otherwise, some loads need to re¬ 
duce. The reduced load of end-user i is denoted by Ri G 
[0, Li]. The objective function of each end-user i is given 
by/i(-Ri) = CiRi+p{lJy^^{L-R)){L^-Ri)-Ui{Li-Ri). 
The quantity CiRi represents the disutility induced by 
load reduction wither > 0. The scalarp(l^(L —i?)) is 
the charged price given the total actual load 1^(T — R). 
The value Ui{Li — Ri) stands for the benefit produced 
by load Li — Ri. 


Each end-user i aims to minimize its own objective func¬ 
tion fi{Ri) by adjusting Ri. Such decision making is sub¬ 
ject to the physical constraints of the power grid. The 
first constraint is that the total actual load cannot ex¬ 
ceed the maximum available supply; i.e., 

irv4(^-^)<irvd^- (43) 


In game (45), p is the pricing policy of load serving en¬ 
tity (LSE). This is confidential information of LSE and 
should not be disclosed to end-users. So the objective 
function fi is partially unknown to end-user i. In addi¬ 
tion, the numerical values of generation and load shift 
factor matrices are of national security interest and are 
kept confidential from the public. Therefore, the power 
flow constraints are unknown to end-users. Assume that 
all the end-users can communicate with LSE. Given ii, 
LSE broadcasts the price valuep(l|^^ | (L—R)), the power 

imbalance 1^^|(L — R) — and power limit vio¬ 
lations HgS - Hi{L -R)- and -HgS + Hi{L- 

R) — In this way, end-users can access the values of 
f., and Gbl in Algorithm 1 without knowing their struc¬ 
tures [f] 

We choose Ui{Li — Ri) = —^ai{Li — Ri)^ + bi{Li — 
Ri) with tti > 0. Eor the pricing mechanism, we use (4) 

in [10]; i.e., p{w) = with (3 > 0, and rj G (0,1). So 
fi{R) = CiRi + {Li ~ Lli)f3{1-f\i^\{L — R))’’ + ^ai{Li — 

Ri)^ — bi{Li — Ri). One can compute the following: 


dm, 

dRidRj 


= a,-2/3i(lfy^|(L-i?))^-i 
- d{L, - R,)^i^ - l)(lfv,|(i - R))--\ 

-^(L,-i?,)^(^ -l)(lfy^|(L-i?))^-2. 

rj fj 


Another set of constraints are induced by the limita¬ 
tions of power flows on branches. Let iLg G [—1, 

(resp. Hi G [—1, be the generation (resp. load) 

shift factor matrix. Eor Hg, the {i,i) entry represents 
the power that is distributed on line £ when IMIV is 
injected into bus i and withdrawn at the reference bus. 
Denote by the maximum capacity of branch e. The 
power flow constraint for branch e can be expressed as: 

_/max < HgS - Hi{L - R) < (44) 

where ^ col[/““]eG£„ ^ = col[5,],ev., L 4 

col[Li]igVi and R = col[i?i]jgVr 


Choose Oi such that Oi > {N — 2)/3 i(lf^,|L)(^-i + (iV- 

l)j3Li^{^ — 1)(1|^^|L)^“^ > 0. Then the monotonicity 
property holds. Assume that there is a global Slater vec¬ 
tor. Eigure 1 shows the simulation results for the IEEE 
30-bus Test System [1] and the system parameters are 


^ Many networked engineering systems operate in a hierar¬ 
chical structure; e.g., Internet, power grid and transportation 
systems, where a central system operator is placed at the top 
layer and end-users are placed at the bottom layer [11,34]. 
Here we assume that LSE can communicate with all the end- 
users. This assumption is widely used in; e.g., network flow 
control [22]. 
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adopted from MATPOWER [2], The delays at each it¬ 
eration are randomly chosen from 0 to 10. 

6.2 Algorithm 2 

Consider 10 players and their components functions are 
defined as follows: 

where 

• Xi = [i/:™ with tpY^ = sample([—10 — 5]) and 

= sample ([5 10]); 

• QW S with = 1 and for j ^ i\ 

• gW S 1R^° with = sample([—50 50]) for j € V. 

One can verify that p = ^ and Lq^ = 2 in Theorem 4.2. 
By (B4), we have pb > 1. In the simulation, we choose 
Pb = 3. Figure 2 presents the estimate evolution of the 
players. 



- playerl 

— player2 

- players 

-player4 

- players 

---players 

player? 

-players 

- players 

. playerlO, 


Fig. 2. The estimates of Algorithm 2 for Tmax = 10. 

7 Conclusions 

We have studied a set of distributed robust adaptive 
computation algorithms for a class of generalized convex 
games. We have shown their asymptotic convergence to 
Nash equilibrium in the presence of network unreliabil¬ 
ity and the uncertainties of component functions and il¬ 
lustrated the algorithm performance via numerical sim¬ 
ulations. 
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Fig. 1. The estimates of Algorithm 1 at 30 buses for Tmax = 10. 
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